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D scription 

The present invention relates to a method for operating a haptic interface unit 
according to the preamble of claim 1, to a respective haptic interface unit, a 
computer program product, and a computer readable storage medium. 

5 Nowadays, haptic interface techniques and haptic interface units become more 
and more popular for increasing the convenience for a user when using and 
operating electronic equipment and appliances. For instance, in the case of 
graphical user interfaces (GUI) haptic interface techniques may support the 
process of operating and selecting different menus. However, present haptic 
10 interface units do not take into account the typical behavior of the different 
users, in particular within a selection process. Therefore, there is still a further 
adaptation necessary to make interface techniques based on haptic interactions 
more convenient for the user. 

15 It is therefore an object of the present invention to provide a method for 
operating a haptic interface unit, which enables a particular simple and reUable 
operation process with respect to a haptic interface unit. 

The object is achieved by a method for operating a haptic interface unit 
20 according to the generic part of claim 1 with the characterizing features of claim 
1. Additionally, the object is achieved by a haptic interface unit according to the 
characterizing features of claim 13, a computer program product according to 
the characterizing features of claim 14. and a computer readable storage 
medium according to the characterizing features of claim 15. 

25 

In known methods for operating haptic interface units at least velocity 
information data with respect to at least one haptic device are generated and /or 
received. Based on and in dependence of at least said velocity information data 
interaction feedback force data are generated and/ or provided which are 
30 descriptive or representative for an interaction feedback force to be generated 
and/ or exerted by said at least one haptic device. Additionally, said interaction 
feedback force data may be transmitted to said at least one haptic device so as 
to generate and/or exert said interaction feedback force. 

35 The inventive method for operating a haptic interface unit is characterized in 
that an inverted damping operation mode is provided. In said inverted damping 
operation mode said interaction feedback force data are at least partly 
generated to be representative for an interaction feedback force which increases 
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With velocity information data being representative for a decreasing velocity, so 
as to generate and/ or exert an interaction feedback force, which increases with 
a decreasing velocity. Additionally or alternatively, said interaction feedback 
force data are at least partly generated to be representative for an interaction 
5 feedback force which decreases with velocity information data being 
representative for an increasing velocity, so as to generate and/ or exert an 
interaction feedback force which decreases with an increasing velocity. In each 
case, said velocity is a velocity with respect to a respective haptic device or a 
pointing unit thereof. 

10 

It is therefore a basic idea of the present invention to generate an interaction 
feedback force which increases with decreasing velocity and which decreases 
with increasing velocity with respect to a haptic device or a pointing unit 
thereof. This concept is referred to as reduced damping or inverted damping 
15 which is counterintuitive with respect to usual handUng of pointing devices, in 
particular haptic devices or the like. 

According to a preferred embodiment of the inventive method, the inverted 
damping operation mode may be performed with respect to each vectorial 
20 components of said interaction feedback force and/ or with respect to said 
velocity, in particular in an independent manner. Consequently, the different 
spatial directions may be evaluated independently from each other, thereby 
giving a more realistic feeling for the user when operating a haptic interface 
unit and when using a respective electronic device or equipment. 

25 

With respect to the respective velocity said interaction feedback force data may 
be generated so as to build up an interaction feedback force which supports and 
may increase a velocity or which may counteract against said velocity. It is of 
particular advantage when said interaction feedback force data are generated to 
30 describe said interaction feedback force as a damping force, so as to generate 
and/or exert an interaction feedback force acting against a given velocity or a 
vectorial component thereof, in particular in the sense of a counterforce or of a 
frictional force. 



35 Additionally or alternatively, the interaction feedback force data are generated to 
describe said interaction feedback force or a vectorial component thereof as 
having an absolute value f being - at least piecewise - a positive monotonically 
decreasing function g of the respective velocity v or of a vectorial component 
thereof, in particular fulfilling the relation 

40 

/(V)ocg(v) . 
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A large variety of at least piecewise positive and monotonically decreasing 
functions g may be chosen. 

For instance, said at least piecewise positive and monotonically decreasing 
5 function g may be chosen to fulfill at least the relation 



where h is an at least piecewise positive and monotonically increasing function 
10 of the velocity v or of a vectorial component thereof. 

Alternatively or additionally, said at least piecewise positive and monotonically 
decreasing function g may be chosen to fulfill at least piecewise the relation 



where v denotes a velocity or a vectorial component thereof. 

Further alternatives are given by choosing the at least piecewise positive and 
20 monotonically decreasing function g as a step function, a staircase function, a 
linear function or the like. 

According to a further alternative of the inventive method for operating a haptic 
interface unit said interaction feedback force data are generated to describe said 
25 interaction feedback force as a force which is at least piecewise dependent on a 
position or a position vector. Therefore, the interaction feedback force may be 
modulated by an underlying spatial structure. 

Here, said position or position vector may be chosen to describe or may be 
30 assigned to a position of a respective haptic device or of an element thereof, in 
particular of a pointing unit or device thereof. 

Alternatively or additionally, said position or position vector may be chosen to 
describe or may be assigned to a position of a corresponding abstract pointing 
35 means within a data structure, in particular of a graphical user interface or the 
like. 



g(v) = 



15 



g(v) = -j^ • 
|v| 
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As a further alternative or addition a holding force mode may be provided, in 
which the absolute value f the interaction feedback force or of a vectorial 
component thereof is increased to a predetermined value fhow or above a 
predetermined force level fmax. if the respective velocity or a vectorial component 
5 thereof decreases below a given threshold value Vn^n, in particular in position 
dependent form. According to this measure a so-called fingerhold mode may be 
realized. 

Further alternatively or additionally, the absolute value f of the interaction 
10 feedback force of a vectorial component thereof may be decreased to a 
predetermined value f^^, in particular of zero, or below a predetermined force 
level fjnin. if the respective velocity v or a vectorial component thereof increases 
above a given threshold value v^ax- 

15 It is a further aspect of the present invention to provide a haptic Interface unit 
which is capable of performing or realizing the inventive method for operating a 
haptic interface unit and/or steps thereof. 

Further additionally, the present invention provides a computer program 
20 product, comprising computer program means adapted to perform and/or 
realize the inventive method for operating a haptic interface unit and /or the 
steps thereof, when it is executed on a computer, a digital signal processing 
means and/or the like. 

25 Finally, the present invention provides a computer readable storage medium 
comprising the inventive computer program product. 

These and further aspects of the present invention will be more elucidated 
taking into account the following remarks: 

30 

A basic idea of the present invention is to provide a haptic interaction technique 
that relies on the counter-intuitive notion of reducing damping forces as the 
user's finger/hand motions speed up, and increasing those forces as the speed 
is reduced. 

35 

Motivation 



In a large range of tasks involving man-machine interactions, a user would have 
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to select an item from a list of items presented by the system, such as a query 
result from a database, or the selection of names from an address list. In these 
cases the items are usually partially ordered according to one sorting criteria 
such as the alphabetical order of names. However, the order of the items does 
5 not allow the user to move straight to the item he is looking for, because the 
distribution of items in the list is not known a priori. As a result, the user has to 
rely on the visual feedback provided by the system displaying the items on the 
list. The coordination between interpreting the visual feedback given by the 
system and the users haptic input for navigating vylthin the list constitute a 
10 closed-loop system. 

In another example - the selection of parameter values for an audio studio mixer 
- the distribution of values is known by the user beforehand, but the overall 
impact of the chosen values on the resulting sound has to be judged by the user 
15 case by case. Again, the resulting interaction paradigm can be viewed as a 
closed-loop system (between haptic input and audio feedback). 

The motivation behind the techniques described in this invention report is to 
help the user perform the selection task by adding haptic feedback to the input 
20 devices used for navigation in large lists of items. 

Concepts 

Based on observing subjects during usability tests who performed a selection 
25 task using a position based control mechanism for choosing values out of a 
known value range, the author of this report discovered that the speed of finger 
movement during the task was a reliable indication of the user's intention 
regarding the selection task. The slower the movement the closer the user was to 
the chosen target in the list. As a result the following three basic interaction 
30 techniques have been combined to form an efficient and enjoyable way of 
navigating in data sets: 

1. Inverted damping mode : A reaction force is applied to the input device that 
is directed opposite to the direction of the user's finger/hand movement. The 
35 strength of the force is inverse proportional to the speed of the user's 

finger/ hand movement. That means, the faster the user moves his hand the 
lower the resisting force will become until it disappears all together (see 
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figure 1 for various functions to implement such an effect). In comparison, a 
well known effect from the real world, viscosity, can be described by a 
reaction force that opposes the users finger/ hand movement, but where the 
strength of the force is proportional to the speed of the users movement. 
Examples for the inverted damping mode are shown in middle sections of 
figures I A to IC, as described below in more detail. 

2. Finger hold mode : As the user's finger/hand movement slows down and goes 
below a certain threshold, the speed based damping forces will be replaced 
by a force that holds the users finger/hand in place. The underlying model 
for this mode is a spring damper model. The user can break out of this mode 
by applying a force to the input device that is larger then a preset force 
threshold set by the system. By doing so the system will transition into the 
third mode, described below. 

3. Force well mode : As long as the speed of the users finger/ hand movement 
stays below a preset threshold, the system will apply forces to the input 
device that oppose the users finger/hand movement, but where the force is 
modulated by the values of the underlying data set. 

For example, if the data set to be manipulated consists of items in a list, 
then the user will feel an increased force whenever he crosses the boundary 
between one item in the list and the neighboring one. In other words, the 
force wells can be considered finger holds for each item in the list, as is 
shown in figure 2. As the speed of the users finger/ hand movement 
increases beyond a given threshold, the system switches back to the inverted 
damping mode. 

Figure 3 below shows the transitions between the three modes as the user 
interacts with the data set. Please note that the decision for a state 
transition is either based on the velocity v of the user's finger/ hand 
movement, or the reaction force f imposed on the user's hand by the spring 
damper model deployed in the finger hold mode. 

An alternative method can be considered as well. Instead of moving from 
"inverted damping" to "finger hold" one can implement the alternative of 
transition from "inverted damping" to "force well" mode and then to "finger 



MULLER . HOFFMANN & PARTNER - 7 - 

Sony International (Europe) GmbH 54.660 31.10.2002 



hold" mode. 
Implementation 

5 First, an implementation suited for desktop system is described that uses a 
general-purpose robotic arm - referred to as the PHANToM - for simulating force- 
feedback input devices. Second, a possible design for an input device based on 
magnetorheological fluids is discussed. The descriptions focus on devices where 
the input values are set by lateral movement of the users hand. An equivalent 
10 method using rotational movement is covered as well in less detail. 

An implementation may create a virtual input device that exhibits the behavior 
of the above mentioned haptic interaction techniques. The PHANToM is 
constrained to movements within a narrow bounding box that allows for only 
15 one degree of freedom for the movement of the robot arm. The full range of 
movement is defined by the end points of the bounding box. By extending one of 
the classes of the respective framework, the inverted damping effect is 
implemented as follows. 

20 We override on method of that class, which is called each time the forces for the 
PHANToM are to be updated. We retrieve the current speed of the PHANToM and 
calculate the new force by multiplying the current speed by a negative constant 
that we chose from a lookup table. Alternatively, a linear or non-linear function 
that relates the speed to the resulting force in the above mentioned fashion can 

25 be applied, compare with figure 1. The finger hold mechanism is implemented 
through the use of a special constraint class provided by the respective 
framework. Again, through monitoring the actual PHANToM reaction force level, 
we decide on stopping the finger hold effect and switch to the force well mode. 
The force well mode can either be implemented through the use of the "Slider" 

30 class of the respective framework, or through the modulation of the inverted 
damping force. As the speed of the user*s finger/hand movement exceeds a 
preset threshold, the inverted damping mode is set again as explained above. In 
case of using a rotational motion for selecting values from a range with inverted 
damping engaged, different classes from the respective framework can be used - 

35 either a dial class or a manipulator class. 

It should be noted, however, that this example is just one possible way of the 
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present embodiment. Using motors as in the PHANToM is also feasable and 
sometimes preferable, such as in studiomixers, where these motors are already 
built in. 

5 A traditional push button or a rotary dial would be augmented with a damping 
unit containing a magnetorheological fluid. By applying a magnetic field to the 
damping unit, the suspended iron particles in the fluid are aligned and as a 
result the viscosity of the fluid changes. The speed of movement can be 
measured by an accelerometer attached to the push button or rotary dial. With 
10 both the accelerometer and the magnetic field under computer control, the above 
mentioned algorithm can be implemented either on a micro controller or as an 
application on the host computer. As potential applications among the 
applications that could benefit from this invention the following seem to be of 
particular importance: 

15 

1. Studio audio mixer: Quick and accurate adjustment of parameters with 
sliders. 

2. Tuning of radio stations: Efficient search with rotary dials {including content 
20 based feedback such as those based on reception quality as well as those 

based on user's bookmarks). 

3. Phone systems with address book: Efficient access of entries in phone book 
via rotary dial. 

25 

4. Video editing systems: Efficient search and editing through additional force 
feedback effects based on semantic information such as scene breaks, etc. 

5. Multimedia content (plus any content that can be listed in an ordered 
30 fashion) retrieval systems: Efficient navigation in data base query results. 

6. Portable devices such as wristwatches, wearable computers, PDAs etc. 

In the following the invention will be described in more detail by taking reference 
35 to the accompanying figures. 
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Fig. lA-C demonstrate different velocity dependencies of the absolute value f 
of the interaction feedback force which can be employed in the 
inventive method for operating a haptic interface unit. 

5 Fig, 2 demonstrates a possible positional dependency of the absolute 

value f of the interaction feedback force which can be employed in 
the inventive method for operating a haptic interface unit. 



Fig. 3 demonstrates the interdependencies of different operation modes of 

10 the inventive method for operating a haptic interface unit. 

Fig. 4 schematically demonstrates a possible inventive haptic interface 

unit. 



15 Fig. 5 is a schematical block diagram demonstrating further details of an 

embodiment of the inventive haptic interface unit. 



Fig. 6 is a flowchart elucidating a preferred embodiment of the inventive 

method for operating a haptic interface unit. 

20 

In the sequence of Figs. lA to IC three different velocity dependencies for the 
absolute value f of the interaction feedback force IFF to be generated and 
exerted are shown. In each of the graphs of Figs. lA to IC the velocity v is 
indicated by the abscissa, whereas the ordinate shows the absolute value f of 

25 the interaction feedback force IFF. In each of the three cases of Figs. lA to IC 
the dependency between the minimum velocity v^„ and the maximum velocity 
Vjnax demonstrates the inventive concept of inverted damping forces. Within this 
particular interval between v^„ and Vj^ax the respective absolute value f of the 
interaction feedback forces shows a monotonical decrease with increasing 

30 velocity v. 



Outside the given interval v^im* v^ax. l-^-. for comparable small velocities v the 
absolute value f of the interaction feedback force IFF is set to a relative high 
value fhoid or f^nax* so as to exert a comparable large force or counterforce to be 
35 supplied to an user. This would lead to the impression that for relative small 
velocities below Vn,i„ a finger or another limp is more or less fixed and hold in a 
fixed position. 

For velocities outside the given velocity interval v^^^, v^nax. i e., for velocities v 
40 larger than the maximum velocity Vn^ax. the absolute value f of the interaction 
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feedback force is set to a small value v^^,. for instance. It is set to zero, allowing 
a comparable free and undisturbed movement. 

These explanations also hold with respect to the embodiments of Figs. IB and 
5 IC. The only difference is the velocity dependence within the interval 

In the embodiment of Fig. lA this dependency is realized by a step function or a 
staircase function with the levels of the different steps decreasing as the velocity 
raises from v^m to v^ax- 

10 

In Fig. IB a linear dependence of the absolute value f is given between the 
boundaries of the interval to v^ax- 

In the embodiment of Fig. IC the dependency of the absolute value f of the 
15 interaction feedback force IFF from the velocity v is given by a hyperbolic 
relationship in the sense of a proportionality to 1/v. 

Fig. 2 demonstrates a positional dependency of the absolute value f of the 
interaction feedback force IFF for a given velocity v in the range between v^ and 

20 Vniax of the embodiments according to Figs. lA to IC. The abscissa shows a one- 
dimensional positional coordinate, for instance in the sense of a distance d. The 
indicated values dl to d5 indicate boundaries or item boundaries dl to d5 
between distinct items U to 14, for instance representative for bottoms or 
selection items on a graphical user interface. As a user moves, for instance a 

25 pointing unit or a pointing device over a spatial region, the interaction feedback 
force IFF and in particular its absolute value f is modulated according to the 
slope shown in Fig. 2. That means, that between the different items II to 14 the 
absolute value f increases as the pointer moves towards a given boundary dl to 
d5 of the item II to 14. In contrast, in the interior of each of the items il to 14 

30 the interaction feedback force IFF is comparable small. Consequently, the 
pointer can be freely moved in the interior of each of the items il to 14 and will 
be stopped or decelerated when moving towards the boundaries dl to d5. 

Fig. 3 demonstrates the interdependency of the different operation modes in an 
35 embodiment of the inventive method for operating a haptic interface unit. Here, 
three operation modes are demonstrated, the first of which being the inverted 
damping operation mode according to the present invention, the second one 
being the holding force mode or fingerhold mode as shown in Figs. lA to IC, and 
the preferred one being a force well mode as exemplified by Fig. 2. Starting with 
40 the inverted damping operation mode Ml of Fig. 3 the fingerhold mode M2 is 
entered, if the velocity v decreases below the damping threshold v^^- If the 
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counterforce exerted by the user against the absolute value f of the interaction 
feedback force increases above the fingerhold threshold fhow or f^^ the force well 
mode M3 is entered which can be left if the velocity v again increases above the 
damping threshold v„^„ so as to again enter the inverted damping operation 
5 mode Ml. 

Fig, 4 is a schematical drawing of an embodiment of the inventive haptic 
interface unit 1 . In the embodiment of Fig. 4 the inventive haptic interface unit 1 
comprises a control unit 10 and a haptic device 20 in the form of a robot arm. 
10 Data exchange between the control unit 10 and the haptic device 20 is 
performed by a PCI interface 30. 

Fig. 5 demonstrates by means of a schematical block diagram more details of an 
embodiment of the inventive haptic interface unit 1. Again, the haptic interface 

15 unit 1 is constituted by a control unit 10 and a haptic device 20. The control 
unit 10 comprises a section 11 for a program, which implements an inverted 
damping algorithm. Based upon the processing data evaluation of section 11 
micro controller 12 are controlled to initiate haptic devices 20 to generate and to 
exert respective interaction feedback forces IFF, in particular based on 

20 respective interaction feedback force data IFFD, the respective interaction 
feedback force IFF acting on a given user U by interacting with an end effector 
20-2. Besides the end effectors 20-2 the respective haptic devices 20 are also 
constituted by force generation means 20-1. Said force generation means 20-1 
may be a step motor as shown on the left-hand side of Fig. 5, or a magnetic 

25 resistance fluid-filled assembly as shown on the right-hand side of the assembly 
of Fig. 5. For controlling the forces IFF to be generated by appropriately adapting 
the interaction feedback force data IFFD position information data PID and/or 
velocity information data VID for a position update and a speed update, 
respectively, are obtained from sensing means 20-3, Said sensing means 20-3 

30 may be a position encoder as shown on the left-hand side of the embodiment of 
Fig. 5. or an accelerometer as shown on the right-hand side of the embodiment 
of Fig. 5. 

Fig. 6 demonstrates a preferred embodiment of the inventive method for 
35 operating a haptic interface unit by means of a flowchart. 

After a starting step SO a haptic framework of the inventive haptic interface unit 
1 is initialized in step SI. Haptic key objects with inverted damping are created 
in the following step S2. Then, a haptic control loop S3 is started. The key state 
40 is set to the inverted damping mode in step 84. In step 85 forces to create 
inverted damping effects are modulated. In the following step S6 it is checked on 
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whether or not the velocity v is lower than a given threshold. If this is the case 
the procedure proceeds with the following step S7 and sets the key state to the 
fingerhold operation mode; otherwise, step S6 of checking the velocity v is 
repeated. In step S8 a force observer or sensing means is activated. Then, in 
5 step S9 the fingerhold or holding force effect is starting. In the following step 
SIO it is checked on whether or not the counterforce exerted by the user is 
above a given threshold. If this is the case the key state is set to the force well 
operation mode in step Sll; otherwise, step SIO for checking the counterforce 
exerted by the user is repeated. In step S12 the force sensing means or force 
10 observer is deactivated. Then, in the following step S13 the forces are modulated 
so as to realize a force well structure. In the following step S14 it is checked on 
whether or not the velocity increases above a given threshold. Is this the case, 
the key state is set to the inverted damping operation mode in step S15 and then 
branched back to step S5 to close the loop 

15 



MULLER . HOFFMANN & PARTNER 



- 13 - 



Sony International (Europe) GmbH 



54.660 



31.10.2002 



Ref r nee Symbols 



1 



haptlc interface unit 



10 



15 



20 



10 control unit 

11 program section 

12 micro controller 
20 haptic device 

20-1 force generation means 

20-2 end effector 

20-3 sensing means 

30 interface 

d distance, position 

dl-dS item boundary 

fhoid holding force 

fmin minimum force 

^max maximum force 

IFF interaction feedback force 

IFFD interaction feedback force data 

PID position information data 

V velocity 

▼min minimum velocity, finger hold threshold 

▼max maximum velocity, no-force threshold 

VID velocity information data 

X position 



